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(57) Abstract 

A method for decoding a message embedded in a pattern of pixels. The method includes the steps of determining the pixel values for 
pixels from the pattern of pixels, determining binary values from the pixel values for pixels from the pattern of pixels; and determining the 
embedded message from the binary values. The pixels have a range of pixel values between a maximum and a minimum. The pixels arc 
divided into cells each having glyph cell and background pixels. The binary value of a glyph pixel is determined by the contrast the glyph 
pixel has with its background pixels. The method can be used to decode embedded web-site address from an image with a foreground 
image and the embedded web-site address. 










FOR THE PURPOSES OF INFORMATION ONLY 








Codes used to identify States party to the PCT on the front pages of pamphlets publishing international applications under the PCT, 


AL 


Albania 


ES 


Spain 


LS 


Lesotho 


SI 


Slovenia 


AM 


Anncnia 


FI 


Finland 


LT 


Lithuania 


SK 


Slovakia 


AT 


Austria 


FR 


France 


LU 


Luxembourg 


SN 


Senegal 


AU 


Australia 


GA 


Gabon 


LV 


Latvia 


SZ 


Swaziland 


AZ 


Azerbaijan 


GB 


United Kingdom 


MC 


Monaco 


TD 


Chad 


BA 


Bosnia and Herzegovina 


GE 


Georgia 


MD 


Republic of Moldova 


TG 


Togo 


DB 


Barbados 


GH 


Ghana 


MG 


Madagascar 


TJ 


Tajikistan 


BE 


Belgium 


GN 


Guinea 


MK 


The former Yugoslav 


TM 


Turkmenistan 


BF 


Burkina Faso 


GR 


Greece 




Republic of Macedonia 


TR 


Turkey 


BG 


Bulgaria 


HU 


Hungary 


ML 


Mali 


TT 


Trinidad and Tobago 


BJ 


Benin 


IE 


Ireland 


MN 


Mongolia 


UA 


Ukraine 


BR 


Brazil 


IL 


Israel 


MR 


Mauritania 


UG 


Uganda 


BY 


Belarus 


IS 


Iceland 


MW 


Malawi 


US 


United States of America 


CA 


Canada 


IT 


Italy 


MX 


Mexico 


UZ 


Uzbekistan 


CF 


Ceniral African Republic 


JP 


Japan 


NE 


Niger 


VN 


Viet Nam 


CC 


Congo 


KE 


Kenya 


NL 


Netherlands 


YU 


Yugoslavia 


CH 


Switzerland 


KG 


Kyrgyzstan 


NO 


Norway 


ZW 


Zimbabwe 


Cl 


C6le d’Ivoire 


KP 


Democratic People’s 


NZ 


New Zealand 






CM 


Cameroon 




Republic of Korea 


PL 


Poland 






CN 


China 


KR 


Republic of Korea 


PT 


Portugal 






CU 


Cuba 


KZ 


Kazakstan 


RO 


Romania 






CZ 


Czech Republic 


LC 


Saint Lucia 


RU 


Russian Federation 






DE 


Germany 


U 


Liechtenstein 


SB 


Sudan 






DK 


Denmark 


LK 


Sri Lanka 


SE 


Sweden 






EE 


Estonia 


LR 


Liberia 


SG 


Singapore 








wo 00/11599 



PCT/US99/12376 



ENCODING AND DECODING A MESSAGE WITHIN AN IMAGE 



FIELD OF THE INVENTION 



The present invention relates to techniques for encoding and decoding messages, and 
more particularly to techniques for encoding a message in an image in which the message is 
not decodable by visual inspection. 



BACKGROUND 

Having machine-readable code on the packaging of a product is a good way to 
transmit information related to the product. For example, bar code has been used for many 
years to carry information such as product identification and inventory infiDimation. Such bar 
codes can be optically read to retrieve the information encoded in the bar codes. 

However, bar codes are one dimensional and are limited in the amount of information 
that can be stored. As a result, two dimensional symbology has been developed to increase 
the amount of data stored by such codes. In using two dimensional codes for recording 
information, precise synchronization is needed to read the data fi"om the symbol pattern 
sequentially. To provide orientation for the two dimensional symbology, the techniques of 
encoding often need visually identifiable features such as lines, frames, concentric rings, axes, 
columns or rows of symbols, or the like, that are optically discriminatable from other sjmbols 
and images. Unfortunately, such techniques are less than desirable if the information is to be 
embedded in a visual image because the visually identifiable features are obtrusive to a viewer 
who wants to observe the image without distraction. 

What is needed is a technique for encoding and decoding embedded messages within a 
visual image without obtrusive features representing the embedded message or the 
synchronization or orientation of the data pattern. 
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U.S.P.N. 4,263,504 (Thomas).U.S.P.N. 5,189,292 (Battennan et al). U.S.P.N. 
5,128,525 (Stearns et al.), U.S.P.N. 5,221,833 (Hecht), U.S.P.N. 5,245,165 (Zhang), 

U.S.P.N. 5,329,107 (Priddy et al.), U.S.P.N. 5,439,354 (Priddy), U.S.P.N. 5,481.103 (Wang), 
U.S.P.N. 5,507,527 (Tomioka, et al.), U.S.P.N. 5,515,447 (Zheng, et al), U.S.P.N. 5,521,372 
(Hecht, et al.), U.S.P.N. 5,541,396 (Rentsch), U.S.P.N. 5,572,010 (Petne), U.S.P.N. 

5,576,532 (Hecht), and U.S.P.N. 5,591,956 (Longacre, et al.) are examples of descriptions of 
the use of two dimensional symbology in storing encoded information. The disclosure of these 
cited patents are incorporated by reference herein in their entirety. 



SUMMARY 



This invention provides techniques for decoding a message embedded in a pattern of 
pixels. The technique includes determining the pixel values for pixels from the pattern of 
pixels; determining binary values from the pixel values for pixels from the pattern of pixels; 
and determining the embedded message from the binary values. In another aspect, the present 
invention provides a device to encode an image with a foreground image having an embedded 
message and also provides a device for decoding the image for the embedded message. 

The invention of the present invention can be advantageously applied in coding and 
decoding messages in a foreground image without resulting in obtrusive changes in the image 
that might overly distract a viewer from visually recognizing the foreground image. An 
example is loading a URL address (that has been embedded in a visual image) into a web 
browser. At the present, when a person “surfs the net” and comes across a “hotspot” (which 
is often a photographic display or blue words) hyperlinked to another web site, the person 
would simply use the pointing device (a mouse) and click at the hotspot to move onto the web 
site of that hotspot. There is, however, no simple way to print out the displayed image and 
later use that image to input the URL address of the web site to direct a web browser to 
connect to the web site. The present technique provides a process, as weU as a device to 
enable a web site to be printed to retain the visual image, yet allowing an electromc reader to 




wo 00/11599 



3 



PCT/US99/12376 



read the image and direct the web browser to connect to that web site. The process includes 
receiving light from a display showing a pattern of pixels and determining pixel value of each 
pixel; recovering an embedded URL address by determining binary values from the pixel 
values for pixels; and loading the URL address into the web browser for connecting to the 

5 web site according to the URL address. The present invention can use grayscale images or 
multicolored images for communicating embedded messages in images. Thus, on a visual 
level, images and words can be displayed for a human to appreciate and read, yet on a less 
obvious level, a message embedded in the image can be read by a machine. The image, with 
the embedded message, when expressed in a printed form, can be taken from place to place 

1 0 conveniently. 



BRIEF DESCRIPTION OF THE DRAWINGS 

The following figures are included to better illustrate the embodiments of the apparatus 

1 5 and technique of the present invention. In these figures, like numerals represent like features 

in the several views. 

Fig. 1 shows an embodiment of a grayscale representation of a block of ceUs of pixels. 
Fig. 2 shows an embodiment of a tile having four cells of pixels. 

Fig. 3A shows a cell with 3x3 pixels. 

2 0 Fig. 3B shows a cell with 5x5 pixels. 

Figs. 4 A to 4 1 show embodiments of how pixels of cells encode logical “1.” 

Figs. 5 A to 51 show embodiments of how pixels of cells encode logical “0.” 

Figs. 6A to 61 show embodiments of how pixels of color cells encode logical “1” and 

“ 0 .” 

25 Fig. 7 shows the image of pixels displaying the word “Webstar”. 

Fig. 8A shows a glyph image displaying “Webstar” with an embedded message. 

Fig. 8B shows a glyph map corresponding to the glyph im^e of Fig. 8A. 

Fig. 8C shows a cell map portion corresponding to the glyph image of Fig. 8A. 




wo 00/11599 



4 



PCT/US99/12376 



Fig. 8D shows a grid map corresponding to the glyph unage of Fig. 8A. 

Fig. 8E shows a data map having sync cells and data cells. 

Fig. 9A illustrates in flow chart form briefly the process of encoding an embedded 

message. 

Fig. 9B illustrales an embodiment of the encoding technique in block form. 

Fig 10 shows self-symmetty inchtding glyph cells laid symmetrically around a global 

center cell. 

Fig. 1 1 shows an embodiment with single layer data block cell layout and symmetnc 
data cells. 

Fig. 12A shows an embodiment of a double layer data block cell layout with symmetnc 
sync cells, and synunetric data cells. 

Fig. 12 B shows a forward sync stream with block tag and error correction code and 
Fig. 12C shows a backward sync stream which U a mirror image of the forward sync stream of 

Fig. 12B. 

Fig. 12D shows a forward data stream with message data and error correction code 
and Fig. 12E shows a backward data stream which is a mirror image of the forward sync 

stream of Fig. 12 D. 

Fig. 13 shows an embodiment of double layer data block ceU layout with symmetnc 
sync cells, and asymmetric data cells. 

Fig. 14A illustrated schematically an embodiment of an image derived from serial 

blocks. 

Fig. 14B to 14D illustrate the serial blocks for the image of Fig. 14A 

Fig. 14E shows the resulting block that consists of the serial block of Figs. 14B to 

14D. 

; Fig. 14F shows a schematic representation of an embodiment of four parallel blocks. 

Fig. 14G to Fig. 14 J show the individual blocks of the four parallel blocks of Fig. 14F. 
Fig. 14K is the overall image of the resulting block that consists of the parallel blocks 



of Fig. 14F. 
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Fig. 1 5 shows an example of the codes of a HTML page. 

Fig. 16 shows the appearance of typical conventional “hotspots” for the web sites of 
that HTML page of Fig. 15. 

Fig. 17 shows the appearance of an image \^dth embedded URL sites “hotspots” for the 

5 web sites of that HTML page of Fig. 15. 

Fig. 18 shows a self-symmetry double-layered, sync symmetric and data symmetric 

data map. 

Fig. 19A illustrates in flow chart form briefly the process of decoding and responding 
to an embedded message. 

1 0 Fig. 19B illustrates an embodiment of the decoding technique in block form. 

Fig. 20A shows an embodiment of an apparatus for encoding a message. 

Fig. 20B illustrates an embodiment of a reader for reading the embedded message. 

Fig. 20C depicts an embodiment of the present invention for decoding a printed image 
with an embedded URL address. 

1 5 Fig. 20D schematically illustrates the structure of a reader. 

Fig. 21 A shows a flow chart depicting how a URL address is embedded in a glyph 

image. 

Fig. 2 IB shows a flow chart depicting how a URL address is decoded firom a glyph 

image. 

20 

DETAILED DESCRIPTION 



In one aspect of the invention, the present invention provides a technique to embed a 
message in a visual image without obtrusive features that draw attention fi'om the visual 
25 image. An example of such is embedding a hyperlink address such as a URL address ( web- 
site address) in an image printed on paper. 

As a first illustration, an embodiment of a single color (or grayscale) image with an 
embedded message is described in detail below. However, a person skilled in the art will be 
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V 



i 



able to adapt the techniques of present disclosure readily for detailed implementation for 
colored embodiments. For a single color (e.g., black-gray-white) with a grayscale image, 
the pixels have gray values that range from a minimum pbcel value (e.g., representing black) to 
a maximum pixel value (e.g., representing white). Likewise, in colored embodiments of other 
5 non-black-white colors, ranges of shades of colors can be similarly implemented. 

Single Color Image 

To better illustrate the present invention using a black-white grayscale embodiment, 
the following values are adopted. It is to be understood that other values (e.g., color, sizes, 

1 0 dimensions, values, and the like) can be similarly adopted by one skilled in the art. 

Fig. 1 shows an embodiment of a grayscale representation of a block 100 of cells 1 12 
of pkels. The cells are divided into two dimensional groups, called “tiles” 122. The encoded 
information in an image is represented by symbols, or glyphs, which in the black-white 
embodiment are represented by pixels 110. To convey lo^cal information, the pixels 1 10 are 
1 5 divided in glyph cells (or simply “cells” herein after) 1 12, each of which is a square (n x n) 
array to convey one binary bit value. A person skilled in the art will know that more than one 
cell can be used to convey one binary bit by having each cell representing a component or 
fraction of a bit. Examples include determining the bit as a whole from the sum, the average, 
the maximum, the contrast, similarity, or the like among two or more number of cells. 

2 0 However, for the sake of clarity of description, for this grayscale embodiment, except where it 

is specifically stated otherwise in context, one cell will only denote one bit. A cell can be a 
synchronization (sync) cell 124 or a data cell 126 (see Fig. 2). (In Fig. 1 and Fig. 2 the dots in 
the upper left cell in a tile are included in the figures for assisting the reader of this disclosure 
to identify the sync cells and do not necessarily denote observable differences.) As can be 
25 seen in Fig. 2 and Fig. 3 A, among the pixels in a cell 1 12 (or sync cell 124 or data cell 126), 
one pbcel, the “glyph pbcel” (GP pixel) 116 (see Fig. 3 A) represents the logical value (i.e., 
binary value) of the cell, whereas the other pixels (the “background pbcels”, or “BP pbcels”) 
118 nearby (in this embodiment surrounding the GP) act as the background and facilitates the 
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designation of the primary pixel in encoding as “black,” “gray.” or “white,” depending on the 
pixel values of all the pixels of the cell. The way the designation is performed will be 
describe below. There can be various levels of shades of grayness. For illustration, the 3 x 3 
cell will be used for describing the present invention, although one skilled in the art can easily 
5 im pl e ment other cell sizes (e.g., n = 2, 4, 5, etc.) based on the present disclosure. Also, it is 
not necessary that the cell be a squire. In a 3 x 3 cell, as shown in Fig. 2 and Fig. 3A, the 
background pixels are BPl, BP2, BP3, BP4, BP6, BP7, BPS, and BP9. 

The logical information in the image are represented by logical binary bit values of 0 
and 1 . In the pixels that convey the logical information, the contrast between one pixel versus 
1 0 others (i.e., the GP pixel 1 1 6 versus the BP pixels 1 1 8), e.g., in a cell, represents one bit. 

The pixel value of a pixel (whether a GP pixel or a BP pixel) is a gray value between a 
minimum and a mayimum, for Convention, denoted as between 0 and 255 (where 0 - black, 
and 255 = white). 

As shown in an embodiment in Figs. 1 and 2, the cells 112 are arranged in glyph tiles 
1 5 (or simply “tiles” hereinafter) 122 each containing m x r cells where m and r are whole 

numbers. For illustration, in this embodiment, a tile contains 2x2 cells. However, it is not 
necessary that the tile be a square. Fig. 2 shows a tile similar to one in the block 100 shown in 
Fig. 1. One of the cells in the tile 122 is a sync cell 124 and the rest are data cells 126. The 
logical value of each cell 1 12 is represented by the GP pixel’s contrast with the BP pixels, and 
20 is either a data bit, which represents the message embedded and associated error correction 
regarding the message, or a synchronization bit (sync bit), which controls the interpretation of 
the data, for example, designating the length of the message, where the error correction bits 
are, and the like. Thus, each tile 122 contains one sync cell 124 and (m x r - 1) data cells 126. 
The sync cells 124 provide spatial synchronization to preserve the logical ordering of the data 
2 5 (which are contained in the data cells 126) so that the glyph code can be read and decoded. 

To this end, the sync cells 124 and the data cells 126 are arranged in a predetermined fashion, 
e.g., a regularly spaced positions repeating from tile to tile. Figs. 1 and 2 show the location of 
sync cell 124 relative to data cells 126 in an embodiment. In this embodiment, from the 
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perspective of the viewer the sync ceU 124 is located at the upper left hand comer of a tile 
which also contain data ceUs 126A, 126B, and 126C. In this embodiment, for example, data 
bits are arranged such that the first bit is located in data cell 126A, the second bit in data cell 
126B, and the third bit in data cell 126C in the first tile. The next three data bits will be 
5 located in the data cells 126A, 126B, 126C of the second tile, and so forth. Minor 

synchronization errors can be tolerated by providing error correction capability. It is to be 
understood that such arrangement of sync cell relative to data ceUs is for iUustrative purposes 
only. Other ways of arrangement are possible. For example, single layer self symmetry 
arrangement, as described infra requires no sync cells apart from the data ceUs. 

10 

Encoding Glyph CeUs 

In general, there can be m GPs and n BPs in a ceU where n is larger then m (Fig. 3B). 
Each cell has a cell contrast (CC) which is a ftmction of the pixel values of all the pixels of the 

cell. 

15 

CC = ABS ((GPl + GP2 + ... + GPm) / m - (BPl + BP2 + ... + BPn) / n) 

where in this equation ABS means the absolute function and GPl, GP2 and GPm represent the 
pixel value of glyph pixels, and BPl, BP2, BPn means the pixel value of background pixels. 

20 For simplicity, as shown in Fig. 3A, in the iUustrative 3 x 3 monotone (i.e., black- 

white-grayscale) glyph ceUs, a glyph ceU 126 has one central glyph pixel, GP and eight 
background pixels (BPl, BP2, BP3, BP4, BP6, BP7, BPS, BP9) surrounding the glyph pixel. 

CC = ABS (GP - (BPl + BP2 + BP3 + BP4 + BP6 + BP7 + BPS + BP9) / S) 

25 

where in this equation ABS means the absolute function and GP, BPl, BP2, etc., represent the 
pixel values of the respective pixels. Black has a pixel value of 0. White has a pixel value of 
255. The pixel values, and therefore the cell contrast can vary from 0 to 255. To represent an 
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embedded message, the cell contrast can be changed by changing the value of the GP pixel. 

The value of background pixels, BPl, BP2 ... BP9 preferably do not change, thereby keeping 
low the distortion to the original visual image (foreground image), which determines the pixel 
values of the background pbcels. 

5 To encode a binary bit of “1” one can change the value of the glyph pixel so that the 

associated cell contrast is large, preferably, the largest possible (preferably keepmg the 
background pixels constant, as stated previously). On the other hand, to encode a distal bit 
“0” the pixel value of the GP pixel can be changed so that the cell contrast is small, preferably 
the smallest possible. In this way, the two binary values can be easily distinguished. 

1 0 Furthermore, due to the use of the absolute function, whether the background is dark or light, 
the binary value can be encoded in the same way. Of course, the choice of encoding a “1” and 
a “0” in the above-described way is arbitrary. One can just as easily encode the two bmary 
values in the opposite way. 

One way to accomplish the above mentioned technique of maximizing cell contrast for 
1 5 a “1” and a ceU contrast for a “0” can be as follows. A cell weight (CW) is 

calculated from the background pixels’ pixel values. 

To encode logical values, the following steps can be used; 

A. Define Encoding Threshold (ET). (Note that ET is independent of glyph cell, glyph tile 
2 0 and glyph block.) 

ET = (Black + White)/ 2. 

ET is the average pixel value of Black (whose pbcel value is 0) and White (whose pbcel value is 
255). Thus ET has a grayscale value of 127. 

25 B. Compute average CeU Weight (CW), for use in background reversal, as wiU be described 
later for encoding the GP pixel value. 

CW = (BPl + BP2 + BPS + BP4 + BP6 + BP7 + BPS + BP9) / 8 
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where, in this equation, BPl to BP9 each represent the pixel value of the pixel it represents 
(i.e., the pixel BPl to BP9, respectively). 

C. Compute glyph pixel (GP) value. 

To encode GP data logical One (1), the GP is given a pixel that is the “reverse” of the 
average cell weight. In other words, if the average cell weight is more white than black, then 
the GP is set to be more black than white, preferably black: 

GP = Black when CW > ET or CW = ET 

GP = White when CW < ET 

Thus, preferably, when a glyph pixel reverses in the pixel value to encode a logical 1, it goes 
to the extreme pixel value (i.e., the pixel value is either the brightest 255, or the dimmest 0). 

In this way, the contrast between the pixel value of the glyph pixel and its surrounding after 
encoding a logical 1 is the largest. Thus, the cell contrast CC after encoding a “1” is the 
largest possible. 

Fig. 4A shows a 3 x 3 cell before encoding a GP of logical 1 where the background is 
white. Thus, the background pbcels BPl to BP9 each have a pixel value of 255, as does the 
GP pbcel. Fig. 41 is the legend (key to the grayscale levels) for Fig. 4A to Fig. 4H. Fig. 4B 
shows the cell after encoding a 1 . After encoding, the background pixels BPl to BP9 continue 
to have a pixel value of 255, whereas the glyph pixel GP changes fi'om a white to a black and 
has a pixel value of 0, which is the largest difference possible from the average background. 
Fig. 4C shows a 3 x 3 cell before encoding a GP of logical 1 where the background is black ( 
BPl to BP9 each have a pbcel value of 0). Fig. 4D shows the cell of Fig. 4C after encoding a 
1. The glyph pixel GP now has a pbcel value of 255. Fig. 4E and Fig. 4F respectively show a 
3x3 cell before and after encoding a GP of logical 1 where the background contains gray 
pixels (BP 1, BP4 are light gray, BP7 is gray, BPS and BP9 is dark gray) and black pixels 
(BP2, BP3, BP6, each is black having pbcel value of 0). Thus, the background is more black 
than white on the average. As a result, the glyph pixel GP changes from a black to white to 
encode a logical 1 . Fig. 4G and Fig. 4H show a 3 x 3 cell before and after encoding a GP of 




wo 00/11599 



PCT/US99/12376 



1 1 

logical 1 where the background contains white pixels (BP2, BPS, BP6), light gray pixels 
(BPl, BP4), gray pkels (BP7, BPS, BP9), resulting an average background which is more 
white than black. To encode a logical 1, the glyph pkel GP changes from a pre-encoding 
value of white to black. 

5 To encode the logical 1, the glyph pixel moves to a post-encoding pbcel value that is as 

far from average background pixel value as possible. However, to encode a data bit of zero 
(0), the GP pkel value is kept near to that of the average background. In one embodiment, 
the GP pixel value remains unchanged from the pre-encoding value. This technique works 
well if the GP grayness is not too different from the surrounding pkels to show the foreground 
1 0 visual image, which is usually the case for macroscopic foreground visual image shown by 
small pixels. In these cases, the glyph pkel GP in the center of the cell remains unchanged in 
pkel value to encode a logical 0. The difference between the GP after encoding a logical 0 
and the average background is typically small. Therefore, an unchanged pkel represents either 
a logical 0 or a background pkel. 

1 5 An alternative is to actively set the post-encoding pkel value of the GP to be near, 

preferably the same as the average background. In this preferred mode, the difference in pkel 
value between the average background and the GP after encoding a logical 0 is zero. 
Comparing this zero difference with the difference of at least 127 in pkel value for encoding a 
logical 1, one can readily tell whether a GP represents a lo^cal 0 or a logical 1. This method 
20 will function regardless of how different the GP is from the average background. As examples 
of this technique. Figs. 5A, 5C, 5E, and 5G show four different cells with difference average 
pkel values before encoding a logical 0 and Fig. 5B, 5D, 5F, and 5H show the four cells, 
respectively, of Figs. 5A, 5C, 5E, and 5G after encoding a logical 0. Fig. 51 is the legend for 
Fig. 5 A to Fig. 5H. Each cell has glyph pkel GP and background pixels BPl, BP2, BPS, 

25 BP4, BP6, BP7, BPS, and BP9. In Fig. 5A and 5C, the background pixels BPl to BP9 are all 
gray. In Fig. 5A, the pre-encoding GP is white and in Fig. 5C the pre-encoding GP is black, 
both of which are changed to gray after encoding a lo^cal 0 (see Fig. 5B and 5D). In Fig. 

5E, some of the background pixels are black whereas some are in different shades of gray. 
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having an average value of gray. As shown in Fig. 5F, the GP is se.t from a pre-encoding 
black to a post-encoding value of gray to encode a “0.” In Fig. 5G the background pixels 
consist of gray, dark gray, and black for an average of dark gray. As shown in Fig. 5H the GP 
is changed from a light gray to dark gray to match the average background to encode a logical 
5 “0.” 

From the above illustrative examples, it is to be understood that a variety of 
background and glyph pixel values can be used in the above scheme of encoding. For 
example, the pkels can have any shade of gray and each of the pixels in a cell can have a 
different shade of gray. As long as the average background pbcel value can be determined 
1 0 (calculated), the logical 1 can be encoded by reversing whereas the logical 0 can be encoded 
by maintaining the pkel value of the GP near that of the background. Using the above 
technique, a message can be embedded in a visual image in which the embedded message is 
neither conspicuous nor decipherable by visual inspection but which can be decoded by 
analyzing the relationship of the pixel values of the pixels in the image. 

1 5 Although in the above-described embodiments, one cell represents one binary bit 

logical value, it is imderstood that one can construct “supercells” in which there are two or 
more glyph pixels adjacent to background pixels in each cell. In this way, either more 
information can be represented in the same space or the same amount information can be 
represented more robustly. Referring now to Fig. 3B, for example, the cell can be a 5 x 5 
20 square with 25 pixels. There are 4 GPs and 21 BPs. Each of the 4 GPs 1302, 1304, 1306 and 
1308 can represent a separate logical value, or the four GPs can collectively represent one 
logical value more robustly. 

Image With More Than One Color 

25 The technique of encoding and decoding a message in a visual image can be done 

in a colored image, i.e., one that contains a plurality of colors. As an illustration, an image 
made of multiple-color pixels (sometime referred to as “color” pbcels for convenience) each 
having the three primary colors (red, blue, and green) can be used. In this way, a whole range 
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of colors can be obtained to form the desired colors of the image. If one wishes, one can use 
colors different from red, blue, and green and the present technique will also work, although 
the visual image may not be as flexible in showing a variety of colors. In the embodiment of 
tri-color (red, blue, green) pixels, each color can have a brightness scale varying from 0 to 255 
5 in pixel value similar to the grayscale range. 

Similar to the grayscale image, one can define an encoding threshold as the average 
of the brightest and the darkest pkel values. Also similar to the grayscale embodiment, the 
lo^cal bit 0 and 1 can be encoded by setting the glyph pixel color brightness based on the 
average of the pbcel values of the pixels surrounding the glyph pbcel. With three colors, if one 
1 0 wish, one can encode three times as much information as one can with a grayscale scheme, in a 
way analogous to the “supercell” described in the above for grayscale embodiments. For 
example, the red color can be used to encode the first part of the embedded message, the 
green color can be used to encode the second part of the message, and the blue color can be 
used to encode the third part of the embedded message. At decoding, the three parts can be 
15 linked together to form a longer whole message. 

To encode a bit value of logical 1, briefly stated, the glyph pbcel assumes a pbcel 
value that is opposite (i.e., in high cx)ntrast) to that of the average background. In other 
words, it assumes a pixel value that is preferably the most different from the pixel value of the 
average background. In this “reversal,” the red channel, pbcel value (PV) is set to 0 when the 
20 background average (BA) is larger than 127 or is equal to 127; PV is set to 255 when BA is 

less than 127. .^s imilar ly in the blue channel and in the green channel each, PV is 0 when the 
background average (BA) is larger than 127 or is equal to 127; PV is 255 when BA is less 
than 127. Thus, when a glyph pbcel “reverses” relative to the backgroimd in the pixel value to 
encode a logical 1, it goes to the extreme pixel value (i.e., its pbcel is either the brightest 255, 
25 or the dimmest 0). In this way, the contrast between the PV before and after encoding is the 

largest. Using this scheme, the data bit can be encoded. Therefore, with three independently 
controlled colors, up to three bit values can be conveyed by one color pbcel (CP), which as 
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used herein contains one pixel of each color component (red, blue, and green in this 
embodiment). 

As an illustration, Figs. 6A to 6H each show a color cell having color background 
pfacels BPl, BP2, BPS, BP4, BP6, BP7, BPS, BP9 surrounding a central color glyph pixel GP. 

5 Fig. 61 is the legend for Fig. 6A to Fig. 6H. Analogous to single-color cells, as used herein 
concerning color pfacels and color cells, the terms glyph pixel (GP) and background pbcel (BP) 
mean color glyph pixel and color background pixel respectively. In each color pixel is a red 
component, a blue component, and a green component. In Fig. 6A, all the color pbcels are 
yellow and having a PV of (255, 255, 0), representing that the red component has a pixel 
1 0 value of 255, the green component has a PV of 255, and the blue component has a PV of 0. 

To encode a logical 1, the GP reverses from the PV of that of the background. Since the 
background red component has a PV of 255, the GP has a red component of PV of 0. Since 
the background green component has a PV of 255, the GP has a green component of PV of 0. 
Since the background blue component has a PV of 0, the GP blue component has a PV of 255. 
1 5 Thus, after encoding a logical 1, the GP has a PV of (0, 0, 255), which is blue (see Fig. 6B). 

This scheme of encoding can be extended to other pixel values (PV). Fig. 6C 
shows a cell in which the PV of all the pixels are each (0, 255, 255), i.e., cyanic (meaning deep 
green and blue here). Since the background average is (0, 255, 255), the GP after reversal to 
encode a logical 1 has a PV of (255, 0, 0), which is a color of deep red (Fig. 6D). 

20 To encode a data bit value of logical 0, the three color components each attain the 

pixel value of the average background. For example, in Fig. 6E, the background pbcels BPl to 
BP4 each are blue having a PV of (0, 0, 255) and the other background pbcels BP6 to BP9 are 
red (255, 0, 0) . To encode a logical 0, the GP attains a magenta color, i.e., a PV of (127, 0, 
127). Similarly, Fig. 6G (pre-encoding) and Fig. 6H (post-encoding) show the GP whose 
25 background is half green and half red attaining a color of yellow (which is composed of red 
and green) to encode a logical “0.” 

Although not described in detail herein, other combinations of colors of various 



pbcel values can be used by one skilled m the art based on the present disclosure. As a fiirther 
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example, not shown in a figure, the glyph pixel GP is green (0, 255, 0). The background 
pixels include a variety of colors; pixel BPl is yellow, pixel BP2 and BP3 are red, pixel BP4 
and BP6 are green, pbcels BP7 and BPS are deep blue, and BP9 is red, resulting in an average 
background pixel value of (96, 80, 96). As a result, after encoding a logical “0,” the glyph 
5 pbcel GP has a pixel value of (96, 80, 96), which is gray. 

Encoding Glyph Block 

The scheme of encoding logical values of “1” and “0” can be used to encode an image 
of pixels with an embedded message. For example, one may want to embed a URL address 
1 0 (web-site address) in an image such that the image will be readily recognized and understood 
by an observer without much distortion caused by the message. As a more specific example, 
one may want to embed the URL address “http:\\www.webstar.com\” in a displayed image 
that shows the word “Webstar”. Fig. 7 is the image of pixels displaying the word “Webstar” 
(i.e., the foreground image) without any embedded message. After embedding the message 
1 5 according to a method of the present invention, the image becomes one shown in Fig. 8A. 

The images in Fig. 7 and 8A are substantially the same in that the object of the foreground 
image, the word “Webstar” is still clearly recognizable in Fig. 8A. At a distance. Fig. 7 and 
Fig. 8A look the same except for an overall gray tmnt in Fig. 8A. The presence of the encoded 
message is indicated by the presence of dots, i.e., black pixels in the white area and white 
2 0 pixels in the black area. 

Knowing the visual image desired and the message to be embedded in the visual image, 
one can use an encoding process including the following steps. As used herein, a “glyph 
block” is a block, i.e., array of pixels that contains the image for a viewer’s visual recognition 
and the encoded message. Thus, the glyph block (or sometimes simply called as “block” 

2 5 hereinafter) consists of 1) a user-defined foreground string, which conveys the visual image, 
and 2) an encoded data stream that is a representation of Os and Is of the message. The pbcel 
appearance representing the encoded message data is not obtrusive and is unrecognizable by 
casual viewing by a viewer under normal lighting. The size of a block is determined by the size 
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of the foreground string or according to the user’s desire as long as it is larger than the size 
of the foreground string. 

For the illustrative embodiment below, the following dimensions are used: 

Cell Dimension = 3x3 pfacels; 

5 Tile Dimension = 2x2 cells; and 
Block Dimension = m x r tiles. 

The glyph block can be encoded vwth the following exemplary steps: 

1 . Select the glyph block size. Based on the foreground image that is to be shown for 
visual perception, one will know the foreground string size, from which the glyph block 

1 0 dimensions can be chosen. The glyph block dimensions can be selected to adequately display 
the visual image, e.g., “Webstar” for visual recognition and for embedding the message. For 
example, as shown in Fig. 7, the visual image “Webstar” in a 48 point “Times New Roman” 
font can be displayed for visual recognition with a glyph block of 210 x 78 pixels, i.e., 70 
columns and 26 rows with a total of 1820 cells, each of which containing 9 pixels. In a 
1 5 scheme in which there are three data cells to one sync cell in a tile, the total number of block 
columns (counting in the horizontal dimension) is the equivalent of: 210 pixels = 70 cells = 35 
tiles. The total number of rows (counting in the vertical dimension) is the equivalent of 78 
pixels = 26 cells =13 tiles. The block thus contains a total of 70 x 26 = 1820 cells. In an 
illustrative tile structure that contains only three data bits per sync bit, the data capacity of the 
20 block for conveying data information is therefore 1820 x (3/4) bits, i.e., 1365 bits, which is 
170 bytes. The sync capacity of the block for storing data decoding parameters is 1820 x 
(1/4) bits, i.e., 455 bits, which is 56 bytes. 

2. Determine the message to be embedded. Depending on the nature of the message 
the user wants to encode, the proper message is selected, so long as the encoded message size 

25 is a fraction of the size of the glyph block size to allow for adequate error correction. The 

level of error correction affects how large or small the message size can be. For example, the 
glyph block size having a data capacity of 170 bytes is adequate for encoding the Internet 
URL address: “http:\\www.webstar.com\”, which has a message size of 23 bytes. 
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3. Create the block tag. A block tag contains information used for decoding. More 
specifically, the block tag contains information on the block type, block ID, message size and 
error correction method. The information on the block type and block ID are used to identify 
serial and parallel blocks. Information on the message size and error correction method is 
5 used in error correction decoding. In a decoding embodiment, the block tag is fixed in size, 4 
bytes, since block tag contains fixed amount of information. 

In the above “webstar” embodiment; 

Error Correction Size for Data = Data Capacity - Message Size = (170 - 23) bytes 

Error Correction Size for Sync = Sync Capacity / 2 - Block Tag Size = (56/2 - 4) bytes 
1 0 Error correction size for data determines how many errors may occur in the data area and still 
be tolerated. For example, given Reed Solomon Error Correction Code is used and error 
correction size for data is 157 bytes, there could have been up to 157 bytes of error in the data 
area, and the correct data is still recoverable. Similarly, if the error correction size for sync is 
22 b 3 des, there can be up to 22 bytes of error in the sync area, and the correct sync is still 
1 5 recoverable. 

4. Create the synchronization stream (sync stream) by combining the block tag and the 
associated error correction data. The sync stream is a stream of lo^cal Os and Is which can be 
generated as a fimction of the block tag. To protect the block tag firom any possible errors, an 
error correction method, e.g. commonly known Reed Solomon Error Correction Code, or 
2 0 functional specially devised error correction method, can be used: 

Sync Stream = Block Tag + Error Correction Function (Block Tag) 

Techniques for generating and using error correction code in self-clocking glyph codes are 
known in the art, e.g., U.S.P.N. 5,771,245 (Zhang) which is incorporated by reference in 
entirety herein. In a preferred mode according to the present invention, the ^c bits in the 
25 sync stream are arranged so that the sync stream has symmetric characteristics. For example, 
the sync stream has a forward component and a reverse component wherein the forward 
component matches the reverse component in opposite order. This scheme is illustrated in the 
data block cell layout shown in Fig. 13. The location of the sync cells are shown by si, s2. 




wo 00/11599 



18 



PCT/US99/12376 



s3, and the like. The data cells are shown by labels of dl, d2, d3, ^d the like. In this 
illustrative example, the block 13 10 has 13 x 5 tiles, each of which has 2 x 2 cells. The block 
1310 thus has 32 forward sync cells, designated as si to s32, as well as 32 backward sync 
cells, designated as sT to s32‘. The si cell 1301 of the forward sync stream is located at the 
5 first tile at the upper left comer of the block 1310. The si' cell 1302 of the backward sync 
stream is located at the last tile at the lower right comer of the block 1310. The data cells are 
arranged in order starting fi-om the first tile (which contains data cells dl, d2, d3) to the last 
tile (which contmns data cells dl91, dl92, p, where p is a padding cell). Alternative schemes 
of the symmetry technique can also be achieved, for example, more than one matching sync 
1 0 stream portion can be used by locating four sync streams at the comers of the block. Other 
embodiments of symmetry will be described later, see “Self-Symmetry code,” infra. 

5. Generate Data Stream by combining the embedded message stream and the error 
correction stream. One method is to spatially concatenate the embedded message stream and 
the error correction stream (i.e.. Data Stream = Message Stream + Error Correction Stream). 

1 5 For example, in the block shown in Fig. 13, using the concatenating technique, the first data 
bits are the message stream. The error correction stream follows immediately after the 
message stream. One skilled in the art will know that such a concatenating technique is for 
illustrative purpose and various schemes of arranging the message stream and the error 
correction stream can be used. Data from the message stream and the error correction stream 
20 can mingle together as long as the mingling method can be interpreted on reading the image to 
decipher the message. 

6. Create data block by interleaving the sync stream with the data stream. Fill each 
tile first with one sync bit, and the rest of the tile with (n x n -1) data bits. The location and 
the content of the sync bits are known and laid out as described above. Once the data bits are 

2 5 known, since location of the sync bits are also known, the data stream bits can be put in, 

ordered in an arrangement one bit after another, going from top to bottom and left to right 
from one tile to the next from the first tile to the last. In this way, the data bits dl, d2, d3, d4, 
etc., are arranged as shown in Fig. 13 . The example of a tile 13 12 with four cells (e.g., four 
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cells si, dl, d2, d3) is shown in Fig. 13. In this way, the sync bits and the data bits are 
spread quite evenly over the block. This provides an unobtrusive way to spatially 
synchronize the data, enabling a visual image that is substantially homogenous except for the 
desired image to be presented, be it a word, a phrase, or an image of a bowl of fruits. 

5 7. Encode the data block from logical values into the desired visual image to produce a 

final encoded glyph image containing pbcels. The glyph image will show an image 
recognizable in a casual viewing. See, Fig. 8A, which shows the glyph block with the 
embedded message in a visually recognizable image “Webstar”. 

10 An illustrative embodiment of message encoding process which can be used in the 

encoding embedded messages is depicted in flow chart form in Fig. 9A. First, the foreground 
visual image to be depicted and the message to be embedded in the foreground visual image 
are determined. This can be done by selecting a message to be embedded (block 902) and 
determining the foreground string (block 904). Then, the message is encoded in the 
1 5 foreground string to generate a glyph image with an embedded message (block 906). This 
glyph image is displayed, e.g., by printing on a medium such as paper (block 908). 

In more detail, the encoding of an illustrative embodiment showing a foreground 
image of words is depicted in Fig. 9B. Knowing the words and the font of the image of the 
foreground string 912, a visual image 914 is created. From the visual image, the block 
20 parameters 916, such as the block dimensions, are determined. On the other hand, knowing 
the user message 918 to be embedded, and based on the block parameters and the error 
correction method chosen, the data stream and sync stream 920 are created. Arranging the 
data stream and the sync stream 920 according to the block parameters 916 results in a glyph 
data block (data block) 922. Incorporating the logical values of the data block into the visual 
2 5 image will result in a glyph image embedded with the user message 924. 



Self-Symmetry Code 
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As shown in the above, one method to synchronize the data bits is to include a 
sync stream in the glyph block along with the data stream, in which the sync stream is 
symmetric but the data stream is asymmetrical. One advantage of symmetry is to provide an 
extra layer of error protection. Another advantage is the ability to recover block location and 
5 dimension quickly. 

Fig. 10 illustrates an embodiment of such a method. In Fig. 10, the self-symmetry 
code includes glyph cells laid symmetrically around a global center cell 1000. Glyph cell 1001 
(having coordinates of xl, yl) and glyph cell 1002 (having coordinates of-xl, -yl) on 
opposite sides of the center cell 1000 (having coordinates of 0, 0) form a pair of cells in mirror 
1 0 position of each other (about the center cell 1000). Numerically, the logical values of the two 
cells are inverse of each other. Geometrically the locations of the two cells are diagonally 
symmetric about the center cell 1000. The pair of glyph cell encodes one distal bit value. If 
primary bit 1001 is larger than mirror bit 1002, the digital bit value is 1. If the opposite is true, 
then the digital bit has a value 0. The error protection is evident since the primary bit 1001 
1 5 can never equal to mirror bit 1002. If they are equal to one and another, then there is an error 
in either cell 1001 or cell 1002. The ability to find location and dimension of a block quickly is 
also apparent since all symmetric cells are about the center cell 1000 and the symmetry can be 
tested quickly. 

Generally there are three types of block layouts for self-symmetry codes; 1) first 
20 layout; a single layer block with no sync cells; 2) second layout; double layer block with 
symmetric sync cells and symmetric data cells; and 3) third layout; double layer block with 
symmetric sync cells and asymmetric data cells. Sync cells are used to provide a storage space 
for block tag information and further used to synchronize the associated block. Data cells are 
used to store message data and the relevant error correction data. Each double layer block 
25 has an associated block tag, which, as previously stated, contains information such as message 
size, error correction method, block index and block type. Block tags are encoded into the 
S 5 mc streams by error correction coding. Tjqjically, the first type (without sync cell) is used 
for encoding a small amount of user message into a small sized image. In this case, instead of 
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iiRing sync cells to store the block tag, one or more default block tags can be assumed. For 
example, within one default block tag the error correction size equals to half of a block's data 
capacity, and within another default tag the error correction size equals to one fourth of a 
block's data capacity, etc. While decoding, a trial and error approach can be used. If the 
5 decoding of a single layer block fails to successfully decode using the first default block tag, 

then the second default tag, and the third, etc., can be tried. If the decoding Ms to decode 
properly using all the default tags then the block is not decodable. 

Fig. 1 1 shows an embodiment of the first layout: a single layer block. In the 
data map 1 102, primary data cells dl to d87 have mirror cells dl’ to d87’ surrounding a 
1 0 centrally located center cell 1 100 (shown as x). The primary data cells are diagonally 

symmetric relative to the mirror cells about the center cell 1 100. Synchronization is achieved 
by arranging data cells symmetrically aroimd the center cell. This layout of cells can be 
identified by a cell-wise symmetry test (i.e., each cell has a mirror cell). When the image is 
read and such symmetry is found, the beginning of the data stream can be located and the data 
15 in the data stream can be analyzed (i.e., decoded) to reveal the embedded message. In such 
self-clocking codes, the synchronization elements (i.e., the elements that tells a reader the 
spatial synchronization so that the decoding can be maintained in a proper order) are data bits 
that convey the data, which may include the embedded message and the relevant error 
correction data. The data map 1 102 has 25 X 7 data cells. The data map 1 102 has a forward 
20 data stream 1104 (represented by cells dl, d2, d3, ... d87) and abackward data stream 1106 
(represented by cells dl’, d2’, d3’, ... , d87’). The bits of the forward data stream 1104 start 
at the first cell (dl) as the first bit and move forward and the bits of the backward data stream 
1106 start as its “first” bit at last cell (dl ’) and move backward in the data map 1 102. By 
comparing the bits starting fi'om one comer to the bits starting fi-om the opposite comer, the 
2 5 orientation of the data stream can be determined and the decoding can then be properly done 

to recover the embedded message. Further, since the data map 1 102 is diagonally symmetric, 
picking the center cell x and comparing the adjacent cells thereto can also show the orientation 
of the cells. For example, if cell d87 is the mirror image in logical value of cell d87’ and cell 
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d86 is the mirror image of cell d86’, and so on, one can conclude that the data stream starts 
at cell dl and continue the decoding process. With the present technique, the first bit to the 
las t bit of the data stream can be correctly determined without using sync bits that are not part 
of data stream. 

5 The second block type with symmetric sync cells and symmetric data cells is used 

to encode moderate amount of user message. The third type with symmetric sync cells and 
asymmetric data cells is used for encoding a larger amount of user message. With a single 
layer block, the error correction is carried out on message stream only. (Note that the data 
stream is the message stream plus error correction stream.) With a double layer block, the 
1 0 error correction is carried on both the sync stream and the message stream. If the error 
correction to the sync stream fails then there is no need to do the error correction to the 
message stream. Since the sync stream is much smaller than the message stream, the 
associated error correction can be tried more quickly. If the sync stream is recovered 
successfully, the related block tag, block location and dimension are therefore correct. 

1 5 Subsequently, the error correction on message stream needs to be done only once. This 

property enables one to carry out the error correction on the sync stream many times in trying 
to find the correct block tag and the block location without slowing down the overall decoding 
speed significantly. 

A double layered block has both sync cells and data cells. For example. Fig. 12A 
20 shows a double layered block 1210 having both sync symmetry and data symmetry. The block 
has tile- wise symmetry, i.e.; each tile has a mirror tile on the opposite side of the center tile 
1200. Primary tile 1201 and its mirror tile 1202 are diagonally symmetric about the center tile 
1200. Another example of symmetry involves tile 1203 and tile 1204. Cells si to s32 depict 
the forward sync stream whereas cells sT to s32’ show the backward sync stream. Cells dl to 
25 d92 show forward data stream whereas cells dl’ to d92’ show the backward data stream. All 

cells indicated by the letter p are padding cells, occup)nng empty cells to allow the 
completeness of the tiles. For illustration. Fig. 12B shows a forward sync stream with the 
block tag and the error correction code and Fig. 12C shows a backward S 3 mc stream which is 
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a mirror image of the forward sync stream of Fig. 12B. Fig. 12D shows a forward data 
stream with the message stream and the error correction code and Fig. 12E shows a 
backward data stream which is a mirror image of the forward sync stream of Fig. 12D. 

As mentioned previously, Fig. 13 shows a double layered block 1310 with 
5 symmetric sync cells but asymmetric data cells about the center tile 1300. The sync cell x of 
the center tile 1300 is the center sync cell of all the sync cells. Cells si to s32 depict the 
forward S 5 mc stream. Cells sT to s32’ depict the backward sync stream. Sync cell si 
(denoted by cell 1301) has a corresponding mirror sync cell sT (denoted by cell 1302), and so 
forth. Cells dl to dl92 depict the data stream and have no mirror cells, since the data stream 

10 is asymmetrical. Cell 1304 is a padding cell. 

If one chooses to, one can use synchronuation method in which both the sync 
stream and the data stream are asjmnmetrical. With the present disclosure, the use of 
asymmetric synchronization methods is within the skills of one skilled in the art and such 
asymmetric methods will not be described in further detail. 

15 

Decoding Glyph Block 

Given a visual image with an embedded message, one decodes the displayed image 
by capturing the displayed image as a glyph image, transforming the glyph image to reduce the 
significance of the foreground image and finding the logical values represented. 

2 0 From the pixel value of a glyph cell, the logical value of the glyph cell can be 

decoded by reversing the encoding method of “1 " and “0" above. 

To implement the decoding of the glyph block, the following steps can be used. 

1 . Capture the glyph Image in an image reader. For example, the image 
“Webstar” URL embedded grayscale image of Fig. 8A can be read using a two-dimensional 

2 5 CCD or CMOS image reader (a camera, etc.) 

2. Produce the glyph map by pixel transformation. This transforms the gl)T)h 
image (e.g.. Fig. 8A) into a glyph map (e.g., shown in Fig. 8B). In Fig. 8B, white pixels in this 
glyph map represent glyph pixels of logical value 1 . Black pbcels may be glyph pfacels of 
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logical value 0 or background pixels. To create the glyph map, pixel transformation on the 
glyph image is performed calculating the transformed pixel value from the measured pixel 
vdue with an equation. 

To this end, pixel transformation to produce the glyph map from the glyph image 
5 can be accomplished by first computing the value of cell contrast (CC) for each pfacel, i.e., the 
absolute difference of that pixel value from the average of those pbcels surrounding it, (e.g., 
the absolute difference between a GP and its background pixels BP). The CC is calculated for 
all pbcels, including GPs and BPs. For example, in the embodiment in which a GP is 
surrounded on all four sides by background pixels (e.g., as in a 3 x 3 c»ll such as in Fig. 3A), 

1 0 the CC is: 

CC = ABS(GPni - (BP 1 + BP2 + BP3 + BP4 + BP6 + BP7 + BPS + BP9) / 8) 

Where “ABS” means the absolute fimction, GP„i is the measured pixel value of the glyph pixel 
1 5 (GP), and BPl, BP2, BP3, BP4, BP6, BP7, BPS, and BP9 are the measured pbcel values of 

the background pixels immediately surrounding the GP (see Fig. 3A). This results in a glyph 
map shown in Fig. SB, which contains white pbcels, black pbcels, and gray pbcels. It is noted 
that all background pixels will generally have a CC of small value and will appear black. Also, 
due to the encoding method used, logical values of “0” will also appear black in Fig. SB. 

20 White pbcels in Fig. SB represent logical values of “1”. 

It is noted that this method of calculating the cell contrast is applicable in a variety 
of cell sizes and cell configurations as long as the relative position between a GP and the BPs 
is predefined. For example, m a 2 x 2 cell, when the pbcels are placed in a regular pattern 
(e.g., in each cell, the GP is in the upper left hand comer and the background pbcels (BPs) are 
25 at the other comers of the square), each GP is still surrounded by BPs, albeit some of the BPs 
in the above equation may originate from two or more cells for a CC value for a GP. 

Similarly, the equation can be used for calculating CC for other cell sizes. 
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Essentially, in Fig. 8B, if the CC value of a glyph pixel GP is above a certain 
predetermined value, then the GP is a “1” logically. If the CC for the GP is small, then the 
GP has a logical value of “0.” Thus, to determine the logical value of a glyph pixel, the CC 
value of a GP is compared to a decoding threshold DT, which can correspond to the value of 
5 ET (the encoding threshold in encoding). For example, if the ET is the average pixel value of 
a darkest pixel (pixel value of 0) and a white pixel (pixel value of 255), then DT can be the 
average between the whitest value and the darkest value in the pixels in Fig. 8B. The logical 
value, i.e., bit value (BV) of a glyph pixel GP is: 

BV= 1 when CC >DT or CC = DT 
10 BV = 0whenCC<DT. 

These bit values (BV) form the normalized transformed pkel value of the GP, and result in a 
map with logical “1" and “0” portions. 

This scheme of decoding is illustrated in the following example. For example, in a 
glyph cell similar to the cell in Fig. 4B, a black, central pkel GP is surrounded by white 
1 5 background cells BPl, BP2, BPS, BP4, BP6, BP7, BP8, md BP9. After transformation, the 
central pkel GP will be white and the background pkels will be black. After calculation, BV 
will be determined to be 1 . This is exactly the logical value encoded by the aforementioned 
encoding technique for Fig. 4B. Likewse, in a glyph cell similar to Fig. 4D, a white central 
cell GP is surrounded by black background cells BPl, BP2, BPS, BP4, BP6, BP7, BP8, and 
2 0 BP9. After transformation, the white central GP remains white, whereas the black 

background BPs remain black. Again, in this case, CC is bigger than DT. Thus, BV is also 1 . 
Similarly, glyph cells such as those shown in Figs. 4F, 4H, 5B, 5D, etc., can be transformed 
and their CC values compared to DT for determination of the logical values. 

In embodiments with multicolored pkels, in analogous manner, the pkels can be 
25 transformed and the GP logical values calculated, by finding whether the value of CC versus 
that of DT for the various colors. 

3. Create the cell map using a two dimensional histogram to determine the cell size. 
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To create the data map (Fig. 18), in which all the GP logical values are known, 
the location of the GPs must be precisely determined. To do this, a cell map showing the 
cell dimensions and the cell configuration (i.e., the relation between neighboring cells) and a 
grid map showing the grid along which GP pixels are lined can be found. The following 

5 method can be used. 

(A) Find all local maxi mum pixels in the glyph map. A “Local Maximum Pixel” is a 
pixel whose pixel value is larger than or equal to (>=) the pbcel value of all adjoining pfacels in 
any matrix of pixels, e.g., 3x3 pixels, not limited to any single cell. As will be described in 
more detail below, local maximum pixels are used for determining the cell dimensions in 

1 0 decoding the information in a glyph block. A “Local Average” is the average pixel value of 
any 3 x 3 pixels. A local average has a value between 0 and 255 for pixels in Fig. 8B. A 
mathematical algorithm can be formulated and a computer method can easily be implemented 
to compare the pkel values in the glyph map to find the local maximum pbcels by one skilled in 
the art, for example, by using the pixel values of Fig. 8B. 

1 5 (B) Select a local window size w x w such that the window is wide enough to 

cover a number of cells. For example, a w value of {w = 2 x (maximum cell size) + 1 } can be 
used; and if one assumes the maximum size of a cell is 6 x 6 pixels, then w = 2x6+l = 13. 

A “local window” is a window of w x w pbcels with the local maximum pbcel being the center 
of each window. All local maximum pixels should have a local window. 

20 (C) Create a two dimensional histogram of size w x w, e.g., a 13 x 13 histogram by 

stacking (i.e., summing) all pixels values of all local windows to the histogram. For 
convenience, the normalized transformed pixel values of logical “0" and “1" can be used. It is 
understood the non-normalized pixel values of Fig. 8B can be used to form the histogram and 
the cell size can still be determined by searchmg for the maximums in the histogram. 

2 5 (D) After all window values are accumulated, the histogram becomes the cell map 

(shown in Fig. 8C) showing the accumulated pbcel values around the neighborhood of the 
local maximum pbcels. Statistically, the large values will represent the glyph pbcels and the 
small values will represent the background pixels. Using the cell map as a template and 
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extending it over the glyph map (e.g.. Fig. 8B), one can find the locations of the glj^ih pixels 
(GPs). 

To determine the cell size, the following method can be used. 

(A) Find the center pixel of the accumulated local windows in the cell map (e.g. 

5 ceU map portion shown in Fig. 8C. The largest number in the map represents a GP, e.g., 9 in 
the cell map of Fig. 8C. Find also four other local maximum pixels nearest to the center pixel 
(which can be called the left, right, above, and below glyph pixels, or LGP, RGP, AGP, and 
BGP), which are the bolded numbers in the cell map (the glyph pixels in bold are all 5s in this 
embodiment of Fig. 8C). 

1 0 (B) Compute the cell ^e as the average distance from the center pixel to the four 

neighbor glyph pixels (GP). For example, the cell size in Fig. 8C is (3 + 3 + 3 + 3)/ 4 = 3. 
That is, the cell dimension recovered is 3 x 3 pixels. 

4. Create the grid map using the cell map and the gJ 3 T)h map. To ensure the correct block 
1 5 location and the block dimension within a scanned image, one can determine a grid in which all 
GPs (data or sync if any) are located. The grid map (see Fig. 8D) is created for this purpose. 
Knowing the cell size, for every pixel m the glyph map that is suspected to be a GP pixel, the 
value of the pixel (pj ) is revised by adding to it the four nearest suspected GP pixels to the left 
( Ipj), right ( rpj), top ( tpj ) and bottom ( bpj ) thereof to form an updated (or revised) grid 
20 pixel value, i.e.. 

Pi , revised = Pi + ^Pi + H^i + apj+ bpj. 

This process is repeated for a number of times, e.g. 4 or 5 times. For a GP pixel in 
25 the block, even if it starts with a low pixel value (e.g., for a logical “0” pixel with low contrast 
to the BPs), due to the larger pixel values at local maximums, (i.e., glyph pixels), after a few 
iterations, the pixel value will progressively increase because of the neighboring glyph pixels 
that are bright (i.e., with large pixel value). In this way, a grid of bright pbcels will be formed 
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to represent the block in which the glyph pixels are located. This technique is equivalent to 
inspecting the glyph map to find the contrasting white dots (pixels) and filling in the glyph 
ma p in the black area between the white dots at locations one cell-length apart to determine all 
the glyph pixel locations. The above technique of calculation is only one practical method of 
5 ascertainmg the positions of the glyph pixels, whether in a logical “0” or “1” state. One skilled 
in the art will be able to select other alternative methods. 

5. Generate the normalized logical data map such that the tile size of the glyph block can be 
found and the block tag can be recovered. To create the normalized logical data map (e.g., 

1 0 Fig. 1 8) the following method can be used. Take the coordinates of all local max (i.e., 

maximum) pixels in the grid map (e.g. Fig. 8D), and get the corresponding pixel values fi-om 
the glyph map (e.g., see Fig. 8B). This creates the cell representation of the glyph block, i.e., 
by picking only the value of GPs; all background pixels are dropped. In this way, one can 
distinguish the backgroimd pixels and the GPs of logical “0" in the transformed glyph map Fig. 
1 5 8B although both have the same pixel value in the glyph map. Subsequently, the entire data 
map can be normalized to some predetermined minimum and maximum values. Then, one can 
determine if the block is a single layered or a double layered block by running a symmetric 
test. For a double layered block, the symmetric test will also uncover tile size. 

20 6. Recover the sync stream and the related block tag. For a double layered block with sync 

symmetry, the sync stream is recovered by determining symmetry. In an embodiment with 
center tile, because of forward and backward sync streams are circular symmetric about the 
center tile, the center tile is first identified, and consequently, the forward and backward 
streams are recovered. Since the forward and backward sync streams are numerically inverse 
25 of one another, the binary value (B V) of sync bits can be recovered by comparing forward 
sync bits with backward sync bits. The BV of a pair of sync bits is one if the value of the 
forward sync bit is larger than that of backward sync bit. The BV of a pair of sync bits is zero 
if the value of the forward sync bit is less then that of backward sync bit. The BV of a pair of 
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sync bits is unknoAvn if the value of the forward s)^c bit is equal to that of backward sync bit, 
which means there is an error in one of the sync bits. To recover the block tag embedded in 
the sync stream, one needs to know the relevant error correction size. By design, the block 
tag is fixed in size (four bytes), the error correction size for the sync stream therefore can be 
5 computed by subtracting the block tag size from the sync stream size. With knowledge of the 
error correction size and a pre-determined error correction method, the error correction 
decoding can be carried out. If the error correction decoding is successful, the correct 
location, dimen sion and ordering of the relevant block are definite. 

10 7. Recover the data stream and the embedded user message. For a block with symmetric 

data cehs, the recovery of bit value (BV) of the data stream is similar to that of the sync 
stream. For a block with asymmetric data cells, the recovery of BV is achieved through the 
following steps. 

(A) Compute the local threshold (LT). For example, in a glyph map similar to the 
1 5 one in Fig. 8B, for every tile, if the sync cell = 1, find the nearest tile with the sync of 0; else 

(i.e., if the sync = 0), find the nearest tile with the sync of 1. Then LT is the average pbcel 
value of sync logical 1 and sync logical 0. 

(B) Decode the bit value (BV) of data cells in a tile. 

B V = 1 , if the pixel value of the data cell >= LT 

2 0 B V = 0, if the pixel value of the data cell < LT 

It is noted the value of LT selected based on the threshold used for encoding logical 0 and 
logical 1 in the encoding process. Selecting a threshold as the average of the pbcel value of 
logical 0 and logical 1 is only one embodiment of the present invention. 

(C) Repeat above steps all tiles. A data map results, having the logical values of all 
25 the bits (similar to one shown in Fig. 18). The data map is obtained by mapping the grid map 

over the glyph map to obtain the logical values. It is understood that a person skilled in the art 
will be able to map the grid map on the glyph map to obtain logical values for the cells even to 
compensate for linear or nonlinear distortions. In the embodiment shown in Fig. 18, a self- 
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symmetry double-layers, sync symmetric and data symmetric data nuip is shown. The center 
tile 1801 is marked by four x’s. The embedded message is recovered using the decoded data 
stream. Error correction is performed on the data stream to recover the embedded message. 
This is accomplished by reversing the function that was used in generating the data stream 
5 from the embedded message; 

Embedded Message = function (Data Stream) 

If error correction is successful, then the embedded message can be recovered 
correctly, e.g. Embedded Message = http:\\www.Webstar.com\ 

1 0 Fig. 19A illustrates in flow chart form briefly the process of decoding and 

responding to an embedded message. To decode, a glyph image is first captured, e.g., with a 
2-dimensional reader (block 1910). The glyph image is decoded to recover the embedded 
message (block 1912). To respond, a processor can act upon the decoded, embedded 
message, e.g., by directing a web browser (e.g., MICROSOFT INTERNET EXPLORER, 

1 5 Microsoft Corporation; NETSCAPE COMMUNICATOR, Nestcape Corp., etc.) to connect 
to a URL address which is the embedded message. 

In more details. Fig. 19B illustrates an embodiment of the decoding technique. 
First, the glyph image 1922 is captured. A glyph map 1924 is created by pixel transformation. 
A cell ihap 1926 is created fi"om the glyph map. A grid map 1928 is created based on both 
20 glyph map 1924 and cell map 1926. A main function of a glyph map is to filter out, i.e., 

identify, glyph cells. A main function of a cell map is to recover dimension and orientation. A 
main fiinction of a grid map is to provide anchor points for cells. The anchor points are 
important since only those cells with logical value of 1 are evident in a glyph map. Glyph data 
block (data block) 1930 is created by mapping glyph map 1924, using the cell map 1926 and 
25 grid map 1928 as guide, fi’om image domain to data domain. Sync stream 1932 is recovered 
by searching for symmetry and specific arrangements under various schemes fi"om glyph data 
block 1930. By performing the associated error correction decoding, the message size and 
block parameters 1936 are recovered from the block tag embedded in the sync stream. The 
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successful recovery of sync streams ascertains the correct locations and dimension of the data 
stream 1934. The user message is decoded when the message size and the block parameters 
are available to interpret the data stream. 

Regarding colored pfacels, in decoding glyph cells in color images, one can use a 
5 scheme that is similar to the decoding of grayscale glyph cells in the monocolor technique. 

Briefly, the technique involves finding whether a glyph pixel has a high contrast with the 
background pixel (i.e., whether the glyph pbcel has a value that is different fi'om that of the 
average background for more than half the maximum pixel value). In this decoding scheme, 
the absolute value difference in pixel value between the average background and the color 
1 0 glyph pixel GP is calculated and called “cell contrast (“CC”).” To decode, the maximum value 
of the three color components of CC is found. In decoding, the bit logical value is 1 if the 
TnayitniiTTi (CC) is equal to or larger than 127, which is the average of the brightest and the 
dimmest pkel value in a color. The bit logical value is 0 if the maximum (CC) is less than 127. 

As illustrative examples. Fig. 6B and Fig. 6D will shown how colored ceUs can be 
1 5 decoded. In Fig. 6B, GP is a blue pixel having a pixel value of (0, 0, 255) corresponding to 
the red, green and blue components. The background pixels BPl, BP2, BPS, BP4, BP6, BP7, 
BPS, and BP9 are yellow, each having a pixel value of (255, 255, 0). The average background 
pixel therefore has a pixel value ofi[255, 255, 0). CC therefore has a pixel value of (255, 255, 
255). As a result, the maximum (CC) is 255, indicating that the logical value of GP is 1 . In 
20 Fig. 6D, GP is a red pixel having a pbcel value of (255, 0, 0) corresponding to the red, green 
and blue components. The background pfacels BPl, BP2, BPS, BP4, BP6, BP7, BPS, and BP9 
are cyanic (i.e., blue green), each having a pixel value of (0, 255, 255). The CC therefore has 
a value of (255, 255, 255). Thus, the maximum (CC) is 255, indicating that the logical value 
of GP is 1 . Other colored glyph cells can be decoded in an analogous manner. A person 
2 5 skilled in the art will understand that with such techniques of coding and decoding logical 0 
and 1 using colors, the other features for embedding a message within a foreground visual 
image, as well as the features for retrieving the embedded message can be implemented with 
techniques similar to that for the grayscale embodiments as described above. 
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Multiple Block Visual Images 

The present invention can be used to encode and decode visual images that span 
two or more blocks. For example, sometimes a message to be embedded is too large to be 
5 embedded into a single visual image. In this case, the message can be divided into a number of 
fractions and each can be embedded into a different visual image. These different visual 
images can be read and the embedded messages can be found and linked together to form the 
final, desired message. Fig. 14A illustrates schematically an embodiment of an image 1400 
with an embedded message wherein both the visual image and the embedded message are 
1 0 divided into three contiguous parts 1401, 1402, 1403. Using the three blocks 1401, 1402, 

1403, one can embed, for example, the message data; 

“http;\\www.glyph.com\serial_blocks.html” in the total image 1400. To this end, one can 
divide the message data into three blocks and embed each into a different image of the blocks 
1401, 1402, 1403. Thus, one can embed the data “http:Wwww.gl” in the first block 1401, the 
1 5 data “yph.com\seria” in the second block 1402, and the data “l_blocks.html” in the third block 
1403. As a result, the first image, i.e., the first part (or block) 1401 of the total wsual image 
1400, shows “THREE” (Fig. 14B), the second image 1402 shows “BLOCK” (Fig. 14C), and 
the third image 1403 shows “IMAGE” (Fig. 14D), each with reversal pixels in the foreground 
image revealing the existence of an embedded message. By visual inspection, the three images 
20 in blocks 1401, 1402, 1403 show the total image “THREE BLOCK IMAGE” (see Fig. 

14E). In each block, a tag section can be present to link the block with the other blocks. For 
example, the sync stream in the block can include the tag in the beginning, which is followed 
by the error correction stream. Similarly, the second block and the third also contain tag 
sections to link the blocks together. Once linked together in series, i.e., serially, the total 
2 5 message reads “http;Wwww.glyph.com\serial_blocks.html”. 

Fig. 14F shows embodiment in which blocks have parallel embedded messages, i.e., 
each block has the same embedded message, although the visual image of the individual blocks 
are distinct. For example, in the glyph block 1410 the overall foreground visual image (with 
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embedded message) can be divided into four blocks: first block 1414, second block 1415, 
third block 1416, and fourth block 1417. 

Fig. 14G to Fig. 14 J show the individual blocks of the four parallel blocks of Fig. 14F. 
Fig. 14K is the overall resulting block that consists of the parallel blocks of Fig. 14F. 

5 In this embodiment of parallel technique, the embedded message in each block 1414, 1415, 

1416, 1417 is “http:\\www.glyph.com\tiled_glyph_blocks”. Thus, there is a redundancy of 
information. Reading any one of the four blocks 1414, 1415, 1416, 1417 will provide the 
message “http;\\www. glvph.com\tiled glyph blocks”. If there is any error in any one of the 
block, such as error caused by part of the glyph block 1410 being covered, the redimdancy will 
1 0 enable the correction of the defective information to provide the accurate message. In each of 
the blocks is also a tag section indicating that these blocks are parallel blocks. 

Further, several glyph blocks each with its separate foreground image and 
embedded message can be placed together as a way to present a meaningful related total 
foreground image with related embedded messages. As a further illustration, Fig. 15 shows an 
1 5 example of the codes of a HTML page 1500 containing four dififerent hyperlinked URL 
addresses: (1) “http://www.netshopper.com/”, (2) 
“http://www.netshopper.com/computers.html”, (3) 
“http://www.netshopper.com/monitors.htinl”, and (4) 

“http://www.netshopper.com/printers.html”. This would be an example of a web site for sale 
20 of computers and related products over the Internet. In a traditional web page encoded with 
the HTML program of Fig. 15, the hyperlinked display is typically projected on a monitor as 
the web page 1502 in Fig. 16. A person browsing the web page using a computer will only 
need to click with a pointer (e.g., a mouse) on, for example, the “ Monitors” hotspot to cause 
the person’s browser to connect to the web site “http://www.netshopper.com/monitors.html”. 
25 The same HTML program of Fig. 1 5, can be encoded to be displayed, e.g., on a printed sheet 
of paper, as a web page 1504 shown in Fig. 17. In Fig. 17, each of the glyph blocks include a 
visual image embedded with a web address. Thus, in block 1506 the “Net Shopper’’ image is 
embedded with the message “http://www.netshopper.com/computers.html”; in block 1508 the 
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“Computers” image is embedded with the message 

“http://www.Qetshopper.com/computers.html”; in block 1510 the “Monitors” image is 
embedded with the message “http://www.netshopper.com/monitors.html”. In block 1512 the 
“Printers” image is embedded with the message ““http://www.netshopper.com/printers.html”. 

5 To connect to an Internet web site corresponding to one of the hotspots of Fig. 17, when one 
of the blocks 1506, 1507, 1508, 1510 is read with a reader in according with the present 
invention, the reader causes a processor (such as the computer) to cause a web browser to 
connect to the URL corresponding to the web site. Note that each of the block (hotspots) of 
Fig. 17 contains a clearly recognizable (readable by human eyes) foreground image with 
1 0 noticeable but not obtrusive dots revealing the presence of the embedded message (URL 

address). How big the blocks have to be depends on the quality of the printer and the reader 
Generally nowadays, the present image sensor technology and printers (e.g., 600 dpi) permits, 
for example, the image of Fig. 17 to be read as 12 point Time Romans font readily. With 
commercial oflfeite printing, much finer prints, and therefore smaller images that are needed to 
1 5 cany the same information, are possible. 

The process of printing embedded URLs of HTML pages 

Normally, the URLs of an HTML page are lost after the page is printed. With 
present technique, the URLs of an HTML page can still be present after the page is printed on 
20 paper. The process of printing embedded URLs with live hot spot on paper firom a standard 
HTML page can be totally automatic and transparent to users. There are five steps in this 
process, briefly stated as follows. 1) Parse an HTML page to find all hot spots. 2) Find the 
location and the dimension of each hot spot. 3) Convert each hot spot into proper foreground 
image. 4) Encode the URL address assodated with each hot spot into the relevant foreground 
2 5 image. 5) Print the URL embedded hot spots in place of regular hot spots on paper. There 
are three major advantages of this process. Firstly, HTML pages need not to be changed. 
Secondly, the layout of an HTML page is not altered. Thirdly, the random look dots appeared 
on hot spots signal the presence of embedded URLs. 
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Integration of Encoding, Printing of Image and Decoding 

In the application using the present technique of encoding and decoding, a message 
(which is not generally decodable by visual inspection) is embedded in a foreground visual 
5 image in an unobtrusive way so as to not distort the visual image significantly. Generally, the 
visual image may be printed on a surface of a substrate, such as paper. The visual image with 
the embedded image can then be read, e.g., by scanning the image with a scanner, into a 
processor, such as an electronic computer, e.g., a personal deslctop computer. 

Fig. 20A shows an example of an apparatus for encoding a message into a visual 
1 0 image and displaying that visual image with the embedded message to be read. The encoding 
apparatus 2000 includes a processor 2002 which has the algorithm for encoding. The 
processor 2002 can be an electronic computer, microprocessor, and the like. The processor 
2002 can contain the codes for encoding the visual image, as well as the embedded message in 
accordance with the technique of the present invention. A user inter&ce 2004 is coimected to 
15 the processor 2002 for inputting parameters, data, programs, modifications of programs and 
algorithms, message, edits of images, and the like, into the processor 2002. Examples of the 
user interface 2004 are keyboard, pointer device (e.g., mouse), light pen, voice activated input 
device, and the like. The processor 2002 itself can have memory for storing the algorithms, 
programs, data, etc., or a memory 2006 can be connected to the processor for such storage. 

2 0 The visual image with embedded message(s) can be displayed for visual viewing, as well as for 
impor ting into a reader by a display 2008. Example of the display 2008 include CRT monitor, 
liquid crystal display, printer, and the like. In the case of a printer, a hard copy 2010 with the 
visual image, which includes the embedded message, can be obtained by printing on a medium, 
such as paper. Optionally, a user interface 2004 can be connected to the display 2008 for 
25 controlling the display. Electrical connections can be done by means of cables, wires, and the 
like. Wireless connections between devices can also be done for signal transmission by, for 
example, electromagnetic waves, such as microwave, or infim'ed light signal transmission. An 




wo 00/11599 PCT/US99/12376 

36 

example of using this apparatus is to print on a piece of paper a visual image, e.g., the image 
“Webstar” that has the URL address “http:\\www.Webstar.com\” embedded therein. 

Fig. 20B illustrates an embodiment of a reader for reading the embedded message 
from the visual image, such as one displayed on the display 2008. The reader 2014 includes an 
5 imager 2016 which records the visual image displayed by the display. For example, if the 
display is a printer producing a hard copy, the imager 2016 can be any device, e.g., a scanner, 
CMOS sensor, or a charge coupled device CCD, which converts the pkels (i.e., image 
elements) into electrical signals. A processor 2018 converts the electrical signals into pixel 
values and into digital values of 0 and 1 to result in the data structure of a bitmap. The 
1 0 processor 2018 employs the algorithms for decoding the bitmap data into the sync stream, 
data stream, and eventually decoding the embedded message. The processor 2018 may have 
its own memory, or it may be connected to a memory 2020 for storage of data, parameters, 
programs, algorithms, and the like. The processor 2018 can be an electric computer, 
microprocessor, or the like. Optionally, a user interface 2022 can be used for inputting data, 
15 parameters, programs, edits, and the like into the processor 2018. Examples of the user 
interface include keyboards, pointer, light pen, voice activated input device, and the like. 

Also, optionally, a user interface 2022 can be connected to the unager 2016 for controlling the 
imager. The imager 2016 can further be a device that captures the bitmap image vrith a video 
pickup device such as a video camera, charge-coupled device, and the like. This is particularly 
20 applicable for capturing the image from a light emitting display, such as a CRT monitor. 
Imagers such as scanners, cameras, etc., are well-known in the art. For example, desk-top 
scanners and hand held scanners are commercially available and their technology is well 
known. 

Fig. 20C depicts an embodiment of the present invention for decoding a printed 
2 5 image with an embedded URL address and activating a computer to direct a web browser to 
access the web site according to the URL address. In this embodiment of an Internet access 
system 2030, a remote control reader 2032 can read the embedded URL from a printed image 
2034 from a printed page 2036 and remotely communicate that information to a TV/computer 
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2038, which is connect by a server to the Internet 2040. The remote control reader 2032, for 
example, has a LCD display 2033 displaying the decoded embedded message. The present 
remote controller has an image sensor (or imager) 2016 and can function as the reader 2014 of 
Fig. 20B. Furthermore, the computer 2038 can actually be a processor in a TV set Avith 
5 access to Internet. The remote control reader 2032 can be a remote controller for the TV set 
with additional features for reading coded message in an image as described herein. Typically, 
a remote control reader would have a battery for energy supply and a transmitter for 
transmitting signals of electromagnetic waves to the computer. Additionally, a pointer device, 
such as a mouse can also be incorporated in the remote control reader 2032 to provide control 
1 0 over the computer to run regular computer software and programs. Technologies on 

traditional remote control, Internet connection, web browsing, and pointers are known in the 
art and will not be described in detail here. 

Fig. 20D schematically illustrates the structure of the remote control reader 2032. 
Briefly stated, the remote control reader has a microprocessor 2040 which controls the 
1 5 communication operation and data processing. A remote control key pad inputs information 
to the microprocessor 2040, for example, to initiate the switching on or off" of a light source 
2042 for illumination of the image 2034 on the printed page 2036. A light sensor (such as a 
CCD camera, CMOS camera, photodetectors array, scaimer) and A/D converter (block 2044) 
produces digital electrical signals dependent on the light intensity (and optionally color in the 
20 case of a color camera) of light impinging on the light sensor. The processor 2040 can have a 
memory 2046 connected to it for storing information, data, program, etc. A decoder 2048 
provides the decoding of the signal received by the microprocessor 2040 from the A/D 
converter to identify the URL address. The decoder 2048 contains the program of algorithms 
for decoding the visual image to derive the message embedded in the visual image. The URL 
25 address is communicated to the computer 2038 via the communication chip 2050. The 
communication chip 2050 and the microprocessor 2040 are in communication with a 
nonvolatile memory in the remote control reader 2032. The nonvolatile memory 2032 
provides data storage capacity for reading an embedded URL address from an image on a 
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printed pnge far away from the computer 2038. The remote control reader 2032 can be 
brought close to the computer 2038 later for activating the access of a web browser to the 
URL address. Thus, for example, when the remote control reader 2032 reads a visual image 
“Webstar” with the embedded URL address of Fig. 8A, it transmits data to the computer 
5 2038 to activate the web browser to access the web site with URL address 

“http:\\www.Webstar.com\”. 

An Internet accessing apparatus according to the present invention can decode an 
image having an embedded Internet address to connect to the Internet. As used herdn, the 
term “Internet” includes the world wide web, and also includes network to web sites whose 
1 0 access is available only to certain authorized users. Regardless of the extent of availability, 
the present invention is applicable for an authorized users to access any such sites by reading 
the web site address from an image with an embedded address. The microprocessor 2040 in 
the remote control reader 2032 can be set such that the decoded web-site address is 
immediately and automatically called by the computer 2038 to connect to the appropriate site 
1 5 through the Internet. Alternatively, the web site can be stored as it is read by the remote 
control reader 2032 and later used to call up the appropriate address. The operation of the 
remote control reader is analogous to a person using a mouse to click on the monitor screen a 
hotspot, i.e., hyperlinked web site, to activate the connection to that hyperlinked site. The 
difference is that instead of using a mouse (or other pointer devices) to click a hyperlinked site 
20 on a screen, in the present web-site decoder, the user uses the remote control reader 2032 to 
read or scan the hotspot in the image with the embedded web-site address. 

The encoding apparatus 2000 and reader 2014 and similar equipment can be used 
to encode and decode according to the techniques described above. Fig. 21A shows a flow 
chart briefly depicting how a URL address (the message) is embedded in a glyph image and 
25 displayed. First, a URL of an HTML page is selected as a message to be embedded (block 
2102). For example, to embed a remote link to a web site “Webstar” one may want to 
embedded the message “http:\\www.webstar.com\”. Also, the hot spot of the HTML page is 
selected as the foreground strings (block 2004). For example, the visual image in which the 
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message may be embedded can be the image “Webstar”. Next, the URL address is encoded 
to embed in the foreground strings (visual image) to generate a glyph image (block 2006). 
The glyph image is displayed (block 2008), e.g., by printing on paper or shown on a computer 
monitor screen instead of the regular hotspot. A printer, for example, can be used to print a 
5 hard copy of the URL address embedded image on paper. 

To read the embedded URL address from the glyph image, the process is briefly 
depicted in the flow chart of Fig. 21B. A displayed glyph image, e.g., one printed on paper is 
read by a 2-dimensional reader, e.g., the remote controUer reader 2032 of the present 
invention (block 21 12). The glyph image is decoded to recover the embedded message data 
1 0 (block 2114). Optionally, the information may be sent to a computer immediately to respond 
to the embedded message that has been decoded (block 21 16). For example, the reader may 
activate a web browser to download the HTML page or to switch to the desired web site 
according to the decoded embedded message, i.e., the web site URL address. 

The present invention has been described in the foregoing specification. The 
1 5 preferred embodiment is for illustrative purpose only and is not to be interpreted as unduly 

limiting the scope of the invention. It is to be understood that modifications and alterations of 
the invention, will be apparent to those skilled in the art without departing from the scope of 
the invention. For example, the algorithms or programs can reside in the remote control 
reader, in the TV/computer, in a program storage medium, such as a floppy disk, hard disk, 

20 tape, compact disk, etc. 
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WHAT IS CLAIMED IS; 

I . A device for loading a web-site address into a web browser for connecting to the web- 
site, comprising: 

(a) reader for receiving light from a pattern of pbcels and determining pixel value of 

5 each pkel; and 

(b) processor having a program of code means readable by the processor to recover an 
embedded web-site address from the pattern of pixels, the program comprising: 

(i) code means for determining the embedded web-site address by determining binary 
values based on the pbcel values for pixels from the pattern of pixels; and 

1 0 (ii) code for loading the web-site address onto the web browser for connecting 

to the web-site according to the web-site address. 

2. The device according to claim 1 wherein the reader comprises a light source for 
illuminating the pattern of pbcels to provide light pattern of different pbcel values and the 

1 5 reader comprises an image sensor to record the light. 

3. The device according to claim 1 wherein the code means for determining the embedded 
web-site address divides the pattern of pixels into cells each containing a matrix of pixels to 
determine said binary values. 

20 

4. The device according to claim 3 wherein the code means for determining the embedded 
web-site address determines a binary value of a pbcel via comparing its pbcel value in relation 
to the pixel values of pixels neighboring thereto. 

25 5. The device according to claim 3 wherein the code means for determining the embedded 

web-site address further compares data from pbcels m one area of the hght pattern to data 
from pbcels in another area of the light pattern for symmetry to determine the embedded web- 
»te address. 
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6. The device according to claim 3 wherein the code means for determining the 
embedded web-site address separates pixels in each cell into one or more glyph pixels and 
background pbcels and determines said binaiy values via determining the relationship of data of 
5 a glyph pbcel in one cell to data of glyph pbcel of another cell. 

7. The device according to claim 3 further comprising a computer running the web 
browser and a transmitter for transmitting electromagnetic signals representing web-site 
address to the computer to direct the web browser to connect to the web site. 

10 

8. A method for loading a URL address into a web browser, comprising; 

(a) receiving li ght from a display showing a pattern of pbcels and determining pbcel 
value of each pbcel; 

(b) recovering an embedded URL address by determining bmaiy values from the pbcel 

1 5 values for pixels from the pattern of pbcels; and 

(c) loading the URL address into the web browser for coimecting to the web site 
according to the URL address. 

9. The method according to claim 8 further comprising comparing pbcel value of a pixel m 
20 relation to the pixel values of pixels neighboring thereto to determine binary value of said 

pbcel. 

10. The method accordmg to claim 8 further comprising comparing data from pbcels in one 
area of the li ght pattern to data from pbcels in another area of the light pattern for symmetry to 

2 5 detOTnme the embedded web-site address. 

1 1 . The method accxirding to claim 8 further comprising dividmg the pattern of pixels into 
cells each containing a matrix of pixels to determine said binary values. 
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12. The method according to claim 1 1 further comprising determining binary values of 
binary bits by arranging the pfacels into cells each having a matrix of pfacels, each cell having 
one or more glyph pixels and neighboring background pixels; and for each glyph pixel, 

5 determining the binary value of the glyph pixel based on the extent of contrast of the glyph 
pbcel's pixel value to that of its background pfacels. 

13. The method according to claim 12 wherein determining binary values comprises: for 
ear.ti glyph pixel determining the average pixel value of its backgroimd pixels and calculating 
1 0 the difference between the glyph pixel's pixel value and its average background pixel value; 
and determining the binary value to be a first binary value if said difference is larger than a 
threshold and determining the binary value to be a second binary value if said difference is 
smaller than the threshold. 

15 14. The method according to claim 1 1 whereas determining binary values firrther 

comprising dividing the pixels in cells via transforming the pixel values of the pixels into a 
gl 5 T)h map by substantially removing the foregroimd image. 

15. The method according to claim 14 whereas the step of recovering an embedded URL 
20 address firrther comprising substantially removing the foreground image to form the glyph map 

by determining for all pixels in the block the absolute value of the difference between a pixel 
and its neighboring pixels. 

16. The method according to claim 14 whereas the step of recovering an embedded URL 
25 address further comprising determining cell size by taking pixels of local maximum pixel 

values (local max pixel) and determining a statistical distance from such local max pixels to 
neighboring local max pixels to indicate the distance from glyph pixel to neighboring glyph 
pixels. 
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17. The method according to claim 16 whereas the step of recovering an embedded URL 
address further comprising determining a grid of the location of the glyph pixels by rendering a 
more uniform pattern of pkel value for glyph pixels by iteratively deriving an updated pbcel 
5 value of each local max pixel from a group containing said local max pixel and its surrounding 
local max pixels. 

18. The method according to claim 15 wha-eas the step of recovering an embedded URL 
address by determining binary values further comprising recovering a data stream including 

1 0 da ta of the image and data of the URL address and recovering a synchronization stream 

directing the interpretation of data in the data stream wherein the sync stream interleaves the 
data stream. 

19. The method according to claim 18 whereas the step of recovering an embedded URL 
1 5 address by determining binary values further comprising determining a synchronization stream 

having a forward synchronization portion and a backward sj/nchronization portion in 
symmetry therewith. 

20. A method for decoding a message embedded in a pattern of pixels, comprising: 

2 0 (a) determining the pixel values for pixels from the pattern of pkels; 

(b) determining binary values from the pbcel values for pbcels from the pattern of 
pbcels; and 

(c) determining the embedded message from the binary values. 

25 21. The method according to claim 20 wherein the pbcels have a range of pbcel values 

between a maximum and a minimum and further comprising: 

(a) for each cell, determining the average pixel value of the background pbcels; 

(b) setting the pbcel values of the glyph pixels, comprising: 
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Q to represent a selected binary value, setting the pixel value of the glyph 
pixel to a pixel value significantly below the threshold if the pixel value of the 
average background is higher than that of the threshold and setting the pbcel value of 
the glyph pfacel to be a pkel value significantly above the threshold if the pfacel value of 

5 the average background is less than that of the threshold; and 

(ii) to represent the other binary value different firom the selected binary value, 
setting the pixel value of the glyph pixel to be about equal to the pbtel value of that of 
the average background. 

10 22. The method according to claim 20 further comprising comparing data fi-om pfacels in 

one area of the light pattern to data fi-om pixels in another area of the light pattern for 
^rametry to determine the embedded message. 

23. A method for embedding a message into an initial image, comprising: 

1 5 (a) determining dimensions of a block of pixels for displaying the initial image; and 

(b) determining a data stream having data of the embedded message in conjrmction with 
data of the initial image such that binary values of the data stream can be represented by pixels 
in the block, wherein a resulting image containing the embedded message is substantially 
similar to the initial image that the resulting image has no substantial visually recognizable 

2 0 irregular distribution of pixels caused by the embedded message. 

24. An optically readable two-dimensional coding system, comprising: a two dimensional 
pattern of pixels, wherein the pixels are divided into tiles of cells, substantially all the tiles 
having a synchronization cell neighbored by a plurality of data cells; the data cells containing 

2 5 factual data and the synchronization cells indicating how the factual data are to be interpreted 
based on the position of the data cells; the synchronization cells substantially interleaving the 
data cells throughout the two dimensional pattern of pixels. 
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25. A method for representing binary values of binary bits with a display of pfacels, 
comprising: 

(a) arranging the pkels into cells each having a matrix of pixels, each cell having one or 
more glyph pixels and neighboring background pixels; and 

5 (b) for each glyph, setting the pixel value of each glyph pixel to render a high contrast 

between the glyph pixel's pre-setting pfacel value and post-setting pkel value to represent a 
selected binary value whereas to not render a high contrast between the glyph pixel's pre- 
setting pixel value and post-setting pfacel value to represent a second selected binary value. 

10 26. A method for two dimensional coding, comprising: 

(a) providing a block of pixels to show a foreground image with an embedded message; 

(b) dividing the pixels into tiles of cells, substantially all the tiles having a 
synchronization cell neighbored by a plurality of data cells; the data cells containing factual 
data and the synchronization cells indicating how the foctual data are to be interpreted based 

15 on the position of the data cells; the synchronization cells substantially interleaving the data 
cells throughout the two dimensional pattern of pixels; and 

(c) representing the binary values of the synchronization cdls and data cells by varying 
the pixel values such that the embedded message does not obtrusively distort the foreground 
image Md such that there is no optically readable two dimensional reference structure. 

20 

27. A method for two dimensional encoding a message, comprising: 

(a) providing a block of pixels to show a message stream representing the message; 

(b) dividing the pixels into tiles of cells such that substantially all of the tiles having the 
same number of data cells, and dividing the data cells to include a first group and a second 

2 5 group to show a message stream; and 

(c) representing the bit values of the message stream such that each bit of the message 
stream is represented by the relationship between a binary value of a cell of the first group and 
the a value of a cell of the second group. 
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FIG. 3B 
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FIG. 9B 
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FIG. 11 
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FIG, 12A 
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<HTML> 

<B0DY> 

<PxA HREF= "http : / /www. net shopper . com/ " xBxFONT 
FACE="Garamond" SIZE=6>Net Shopper</Bx/FONTx/Ax/P> 

<PxA HREF= "http : / /www . netshopper . com/computers . html " xFONT 
SIZE=5>Computers</FONTx/A> </P> 

<PxA HREF= "http : / /www . netshopper . com/monitors . html " xFONT 
SIZE=5>Monitors</FONTx/A> </P> 

<PxA HREF="http : //www.netshopper. com/ printers . html "xFONT 
SIZE=5>Printers</FONTx/Ax/Px/BODY> 

</HTML> 

FIG. 15 ^1500 



Net Shopp er 

Comout^s 



Monitors 




FIG. 16 




FIG. 17 



1504 







0000001110000101000100000100000100010011100101010001010001100111000000 



wo 00/1 1599 



18/22 



PCT/US99/12376 




FIG. 18 
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